SEQUENCE LABELING

Ask-and-Verify: Span Candidate Generation and Verification for Attribute Value Extraction (2022.emnlp-industry)

Challenge:	Pre-trained language models (PLMs) construct word representations at subword level with Byte-Pair Encoding (BPE) or its variations . but these methods split a word into subword units and make it incomplete and fragile .
Approach:	They propose a character-aware pre-trained language model to tackle OOV problems . they construct contextual word embedding for each token from sequential character representations .
Outcome:	The proposed model improves on the existing models on multiple NLP benchmarks.

Copied to clipboard

Yifan Ding, Yan Liang, Nasser Zalmout, Xian Li, Christan Grant, Tim Weninger

FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP (N19-4)

Challenge:	Existing reading comprehension models can over-generate attribute values which hinders precision.
Approach:	They propose a product attribute value extraction task that captures key factual information from product descriptions and a new end-to-end pipeline framework called Ask-and-Verify.
Outcome:	The proposed framework outperforms existing models by up to 3.1% F1 absolute improvement points while scaling to thousands of attributes.

Copied to clipboard

Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, Roland Vollgraf

FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction (2022.naacl-demo)

Challenge:	Existing approaches combine word embeddings with character-level features to model additional features such as subword structures and meaning ambiguity.
Approach:	They present FLAIR, an NLP framework that enables embeddings of word and document data . they propose a hierarchical learning architecture that concatenates output states of a character-level CNN or RNN with the output states from a task data.
Outcome:	The proposed framework hides embedding-specific engineering complexity and allows researchers to "mix and match" various embeddables with little effort.

Copied to clipboard

Minh Van Nguyen, Nghia Ngo, Bonan Min, Thien Nguyen

Accelerating BERT Inference for Sequence Labeling via Early-Exit (2021.acl-long)

Challenge:	Existing active learning frameworks require long time between annotation batches due to time-consuming nature of model training and data selection.
Approach:	They propose a small proxy network to synchronize the proxy network with the main large model to ensure appropriateness of the selected annotation examples for the main model.
Outcome:	The proposed framework can support multiple languages and is available on github and demo website.

Copied to clipboard

Xiaonan Li, Yunfan Shao, Tianxiang Sun, Hang Yan, Xipeng Qiu, Xuanjing Huang

A k-Nearest Neighbor Approach towards Multi-level Sequence Labeling (N19-2)

Challenge:	Existing early-exit mechanisms are designed for sequence-level tasks, rather than sequence labeling.
Approach:	They propose to extend sentence-level early-exit to accelerate inference of PTMs . they propose a token-level mechanism that allows partial tokens to exit early at different layers .
Outcome:	The proposed approach can save up to 66%75% inference cost with minimal performance degradation.

Copied to clipboard

Yue Chen, John Chen

Dialogue Systems & Conversational Agents

Challenge:	Existing methods for complex dialog management require limited training data.
Approach:	They propose a method for intent recognition for complex dialog management in low resource situations . they use windowed word n-grams, POS tag n grams and pre-trained word embeddings as features .
Outcome:	The proposed method performs better with less than 1% of the data size than existing methods but requires considerably more data.

Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP (2021.eacl-demos)

Copied to clipboard

Rob van der Goot, Ahmet Üstün, Alan Ramponi, Ibrahim Sharaf, Barbara Plank

Embeddings of Label Components for Sequence Labeling: A Case Study of Fine-grained Named Entity Recognition (2020.acl-srw)

Challenge:	Multi-task learning (MTL) has become a standard repertoire in natural language processing (NLP) it enables neural networks to learn tasks in parallel while leveraging the benefits of sharing parameters.
Approach:	They propose a toolkit for fine-tuning contextualized embeddings in multi-task settings.
Outcome:	The proposed toolkit supports a variety of natural language processing tasks . it enables neural networks to learn tasks in parallel while leveraging the benefits of sharing parameters.

Copied to clipboard

Takuma Kato, Kaori Abe, Hiroki Ouchi, Shumpei Miyawaki, Jun Suzuki, Kentaro Inui

Zero Pronouns Identification based on Span prediction (2021.acl-srw)

Challenge:	In general, the labels used in sequence labeling consist of different types of elements.
Approach:	They propose to integrate label component information as embeddings into sequence labeling models.
Outcome:	The proposed method improves on English and Japanese fine-grained named entity recognition on low-frequency labels.

Copied to clipboard

Sei Iwata, Taro Watanabe, Masaaki Nagata

Challenge:	Pro-drop languages allow omissions of essential phrases or arguments . the presence of zero-pronouns affects downstream tasks of NLP .
Approach:	They propose a query-based method to identify zero-pronoun arguments . they use Japanese and Chinese datasets to evaluate the method .
Outcome:	The proposed method surpasses the sequence labeling baseline on Japanese and Chinese datasets.

LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation (2020.acl-demos)

Copied to clipboard

Dong-Ho Lee, Rahul Khanna, Bill Yuchen Lin, Seyeon Lee, Qinyuan Ye, Elizabeth Boschee, Leonardo Neves, Xiang Ren

GlotEval: A Test Suite for Massively Multilingual Evaluation of Large Language Models (2025.emnlp-demos)

Challenge:	Existing frameworks for sequence labeling and classification require massive human effort and labeling data is limited.
Approach:	They propose a web-based, Label-Efficient AnnotatioN framework that allows an annotator to provide the needed labels for a task and can capture explanations for each labeling decision.
Outcome:	The proposed framework surpasses baseline F1 scores by 5-10 percentage points while using 2X times fewer labeled instances.

Copied to clipboard

Hengyu Luo, Zihao Li, Joseph Attieh, Sawal Devkota, Ona de Gibert, Xu Huang, Shaoxiong Ji, Peiqin Lin, Bhavani Sai Praneeth Varma Mantina, Ananda Sreenidhi, Raúl Vázquez, Mengjie Wang, Samea Yusofi, Fei Yuan, Jörg Tiedemann

Multilinguality

Challenge:	Existing evaluation frameworks focus on English and a handful of high-resource languages, thereby overlooking the realistic performance of large language models in multilingual and lower-resourced scenarios.
Approach:	They propose a unified and lightweight framework that integrates 27 benchmarks under a standard ISO 639-3 language identifier system to enable seamless incorporation of new benchmarks.
Outcome:	The proposed framework integrates 27 benchmarks under a standard ISO 639-3 language identifier system, allowing for seamless incorporation of new benchmarks.

Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor (2021.acl-long)

Copied to clipboard

Xinyu Wang, Yong Jiang, Zhaohui Yan, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

Green & Sustainable NLP

Challenge:	Knowledge distillation is a technique to transfer knowledge between models, typically from a large model (the teacher) to a more fine-grained one (the student).
Approach:	They propose a factorized form of the knowledge distillation objective for structured prediction which is tractable for many typical choices of the teacher and student models.
Outcome:	The proposed model is able to transfer knowledge between teacher and student models without loss of accuracy under four different scenarios.

Overcome Noise and Bias: Segmentation-Aided Multi-Granularity Denoising and Debiasing for Enhanced Quarduples Extraction in Dialogue (2024.emnlp-main)

Copied to clipboard

Xianlong Luo, Meng Yang, Yihao Wang

Robustness in NLP

Challenge:	Existing methods for generating sentiment quadruples in dialogues face heightened noise and order bias challenges, leading to decreased robustness and accuracy.
Approach:	They propose a Segmentation-Aided multi-grained denoising and debiasing method to address noise and order bias challenges in ABSA.
Outcome:	The proposed method achieves word-level denoising and utterance-level demoising via topic-aware dialogue segmentation.

Boundary Detection with BERT for Span-level Emotion Cause Analysis (2021.findings-acl)

Copied to clipboard

Xiangju Li, Wei Gao, Shi Feng, Yifei Zhang, Daling Wang

New or Old? Exploring How Pre-Trained Language Models Represent Discourse Entities (2022.coling-1)

Challenge:	Emotion cause analysis (ECA) is an emerging topic in natural language processing, which aims to identify the reasons behind a given emotion.
Approach:	They propose to detect the precise boundaries of text spans conveying accurate emotion causes from the given context by a sequence labeling and position identification problem.
Outcome:	The proposed methods outperform existing models on two benchmark datasets on the emotion cause analysis task.

Copied to clipboard

Sharid Loáiciga, Anne Beyer, David Schlangen

A Multi-lingual Multi-task Architecture for Low-resource Sequence Labeling (P18-1)

Challenge:	Recent research shows pre-trained language models learn to encode syntactic knowledge to a certain degree.
Approach:	They propose to investigate the information-status of entities as discourse-new or discourse-old . they use binary classification and sequence labeling to investigate their ability to encode syntactic knowledge .
Outcome:	The proposed models encode information on whether an entity has been introduced before or not in the discourse.

Copied to clipboard

Ying Lin, Shengqi Yang, Veselin Stoyanov, Heng Ji

Low-Resource NLP

Challenge:	Existing studies have shown that multi-task learning can boost the performance of related tasks such as MT and abstractive text summarization.
Approach:	They propose a multi-lingual multi-task architecture to develop supervised models with a minimal amount of labeled data for sequence labeling.
Outcome:	The proposed architecture achieves 4.3%-50.5% absolute gains compared to mono-lingual model . the proposed model is particularly effective in low-resource settings .

Viable Dependency Parsing as Sequence Labeling (N19-1)

Copied to clipboard

Michalina Strzyz, David Vilares, Carlos Gómez-Rodríguez

Frustratingly Simple Few-Shot Slot Tagging (2021.findings-acl)

Challenge:	Existing work on dependency parsing by sequence labeling suggested that it was impractical.
Approach:	They propose to use dependency trees as sequence labels to obtain fast and accurate parsers using a conventional BILSTM-based model.
Outcome:	The proposed models are conceptually simple, not needing traditional parsing algorithms or auxiliary structures, and provide a good speed-accuracy tradeoff, with results competitive with more complex approaches.

Copied to clipboard

Jianqiang Ma, Zeyu Yan, Chang Li, Yang Zhang

BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla (2022.findings-naacl)

Challenge:	Existing fewshot methods for slot tagging are weak in encoding slot name semantics and slot dependencies.
Approach:	They propose a simple and effective few-shot model for slot tagging which incorporates machine reading comprehension (MRC) using source domain and target domain data.
Outcome:	The proposed model outperforms state-of-the-art methods on the SNIPS dataset.

Copied to clipboard

Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, Kazi Samin Mubasshir, Md Saiful Islam, Anindya Iqbal, M. Sohel Rahman, Rifat Shahriyar

Robust Transfer Learning with Pretrained Language Models through Adapters (2021.acl-short)

Challenge:	Bangla is a widely spoken yet low-resource language in the NLP literature.
Approach:	They propose a BERT-based natural language understanding model pretrainable in Bangla, a widely spoken yet low-resource language in the NLP literature.
Outcome:	The proposed model outperforms multilingual and monolingual models on four NLU tasks covering text classification, sequence labeling, and span prediction.

Copied to clipboard

Wenjuan Han, Bo Pang, Ying Nian Wu

Local String Transduction as Sequence Labeling (C18-1)

Challenge:	Existing approaches to transfer learning with pretrained transformer-based language models are not robust and can be adversarial.
Approach:	They propose a simple yet effective adapter-based approach to fine-tune language models on downstream tasks.
Outcome:	The proposed approach improves stability and adversarial robustness in transfer learning to various downstream tasks.

Copied to clipboard

Joana Ribeiro, Shashi Narayan, Shay B. Cohen, Xavier Carreras

Visual Data in NLP

Challenge:	String transduction and sequence labeling are often treated as separate entities and often give treatment to different problems in NLP.
Approach:	They propose to reduce string transduction to sequence labeling by using a finite-state technique that uses string transducing and sequence labelling.
Outcome:	The proposed method performs better than seq2seq models and yields state-of-the-art results in several cases.

OptSLA: an Optimization-Based Approach for Sequential Label Aggregation (2020.findings-emnlp)

Copied to clipboard

Nasim Sabetpour, Adithya Kulkarni, Qi Li

DualNER: A Dual-Teaching framework for Zero-shot Cross-lingual Named Entity Recognition (2022.findings-emnlp)

Challenge:	Existing approaches to annotate data are based on binary and multi-choice problems, but little work has been done on complex tasks such as sequence labeling with imbalanced classes.
Approach:	They propose an optimization-based Sequential Label Aggregation method that integrates worker reliabilities and advanced deep learning techniques to conquer the challenge.
Outcome:	The proposed method outperforms state-of-the-art methods and is easier to interpret.

Copied to clipboard

Jiali Zeng, Yufan Jiang, Yongjing Yin, Xu Wang, Binghuai Lin, Yunbo Cao

Ab Initio: Automatic Latin Proto-word Reconstruction (C18-1)

Challenge:	Existing approaches to named entity recognition (NER) are limited to high-resource languages like English and Chinese.
Approach:	They propose a framework to make full use of annotated source and unlabeled target language text for zero-shot cross-lingual named entity recognition.
Outcome:	The proposed framework makes full use of both annotated source and unlabeled target language text for zero-shot cross-lingual named entity recognition (NER).

Copied to clipboard

Alina Maria Ciobanu, Liviu P. Dinu

Challenge:	Existing methods for proto-word reconstruction are time-consuming and manual, but few studies have done it . a recent study used cognates to reconstruct ancient languages from their modern counterparts .
Approach:	They propose to use Latin proto-words to automate the process of proto-language reconstruction . they leverage information from all modern languages and use conditional random fields for sequence labeling .
Outcome:	The proposed method improves on previous results and requires less data . it is based on word forms in multiple Romance languages and on recurrent neural networks .

NAT: Noise-Aware Training for Robust Neural Sequence Labeling (2020.acl-main)

Copied to clipboard

Marcin Namysl, Sven Behnke, Joachim Köhler

Robustness in NLP

Challenge:	Sequence labeling systems should perform reliably under ideal conditions and with corrupted inputs.
Approach:	They propose two noise-aware training objectives that improve robustness of sequence labeling performed on perturbed inputs.
Outcome:	The proposed methods improve robustness on English and German named entity recognition benchmarks.

Towards Making a Dependency Parser See (D19-1)

Copied to clipboard

Michalina Strzyz, David Vilares, Carlos Gómez-Rodríguez

Constituent Parsing as Sequence Labeling (D18-1)

Challenge:	Eye trackers and gaze features collected from them have been recently applied to natural language processing (NLP) tasks such as part-of-speech tagging.
Approach:	They propose to leverage eye-tracking data in an RNN dependency parser when no aggregated or token-level gaze features are used at inference time.
Outcome:	The proposed model can be used to improve performance on non-gazed treebanks.

Copied to clipboard

Carlos Gómez-Rodríguez, David Vilares

Challenge:	Constituent parsing is a core problem in NLP where the goal is to obtain the syntactic structure of sentences expressed as a phrase structure tree.
Approach:	They propose a method to reduce constituent parsing to sequence labeling by using a tree with unary branches.
Outcome:	The proposed method outperforms the Vinyals et al. (2015) sequence-to-sequence parser by 90% on the PTB and CTB treebanks.

An Evaluation of Progressive Neural Networksfor Transfer Learning in Natural Language Processing (2020.lrec-1)

Copied to clipboard

Abdul Moeed, Gerhard Hagerer, Sumit Dugar, Sarthak Gupta, Mainak Ghosh, Hannah Danner, Oliver Mitevski, Andreas Nawroth, Georg Groh

SimCKP: Simple Contrastive Learning of Keyphrase Representations (2023.findings-emnlp)

Challenge:	Fine-tuning suffers from catastrophic forgetting, a problem exacerbated in natural language processing (NLP).
Approach:	They propose to use progressive neural networks to re-use previously learned knowledge when learning new tasks.
Outcome:	The proposed approach improves on common NLP tasks across a range of architectures, datasets, and tasks.

Copied to clipboard

Minseok Choi, Chaeheon Gwak, Seho Kim, Si Kim, Jaegul Choo

WYWEB: A NLP Evaluation Benchmark For Classical Chinese (2023.findings-acl)

Challenge:	Existing models for keyphrase generation and keyphrase extraction use a token level to generate keyphrases that do not appear in a document.
Approach:	They propose a simple contrastive learning framework that generates keyphrases that do not appear in a document and a reranker that adapts the scores for each generated phrase.
Outcome:	The proposed model outperforms the state-of-the-art models on multiple benchmark datasets.

Copied to clipboard

Bo Zhou, Qianglong Chen, Tianyu Wang, Xiaomi Zhong, Yin Zhang

Challenge:	Existing benchmarks for classical Chinese are inadequate to evaluate performance of different NLP models.
Approach:	They propose an evaluation benchmark for classical Chinese NLP, which evaluates existing models.
Outcome:	The proposed benchmark evaluates the performance of existing models in classical Chinese.

Discontinuous Constituent Parsing as Sequence Labeling (2020.emnlp-main)

Copied to clipboard

David Vilares, Carlos Gómez-Rodríguez

Bringing Emerging Architectures to Sequence Labeling in NLP (2026.eacl-long)

Challenge:	Existing approaches to discontinuous parsing are complex and low-level.
Approach:	They propose to encode discontinuities as nearly ordered permutations of the input sequence.
Outcome:	The proposed model is fast and accurate under the right representation.

Copied to clipboard

Ana Ezquerro, Carlos Gómez-Rodríguez, David Vilares

Challenge:	Pretrained Transformer encoders are the dominant approach to sequence labeling . however, few have been applied to sequence labels on flat or simplified tasks .
Approach:	They propose to use pretrained Transformer encoders to model relations across words . they find that the architectures adapt well across tagging tasks that vary in complexity .
Outcome:	The proposed architectures perform well across tagging tasks across languages and datasets.

Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition (2022.findings-emnlp)

Copied to clipboard

Zihan Wang, Kewen Zhao, Zilong Wang, Jingbo Shang

GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling (P19-1)

Challenge:	Existing methods for fine-tuning pre-trained language models are limited . we propose a few-shot fine-uning framework for NER .
Approach:	They propose a few-shot fine-tuning framework for named entity recognition (NER) they propose three new types of tokens, "is-entity", "which-type" and "bracket"
Outcome:	The proposed framework improves on pre-trained language models on several benchmark datasets.

Copied to clipboard

Yijin Liu, Fandong Meng, Jinchao Zhang, Jinan Xu, Yufeng Chen, Jie Zhou

Chunking

Challenge:	Existing systems for sequence labeling are limited by shallow connections between consecutive hidden states and insufficient modeling of global information.
Approach:	They propose a global context enhanced deep transition architecture for sequence labeling . they deepen the state transition path at each position in a sentence and assign tokens with global representations .
Outcome:	The proposed architecture outperforms the best reported results on two standard sequence labeling tasks.

Learning Numeral Embedding (2020.findings-emnlp)

Copied to clipboard

Chengyue Jiang, Zhonglin Nian, Kaihao Guo, Shanbo Chu, Yinggong Zhao, Libin Shen, Kewei Tu

An Investigation of Potential Function Designs for Neural CRF (2020.findings-emnlp)

Challenge:	Existing word embedding methods do not learn numeral embedds well because numerals are limited in number and their appearances in training corpora are highly scarce.
Approach:	They propose two numeral embedding methods that can handle the out-of-vocabulary problem for numerals.
Outcome:	The proposed methods can handle the out-of-vocabulary problem for numerals.

Copied to clipboard

Zechuan Hu, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling (N19-1)

Challenge:	Existing approaches to sequence labeling are based on the neural linear-chain CRF model.
Approach:	They propose a series of increasingly expressive potential functions for neural CRF models that integrate emission and transition functions and explicitly take contextual words as input.
Outcome:	The proposed model consistently achieves the best performance on the decomposed quadrilinear potential function based on the representations of two neighboring labels and two neighbored words.

Copied to clipboard

Peng Lu, Ting Bai, Philippe Langlais

Context-aware Information-theoretic Causal De-biasing for Interactive Sequence Labeling (2022.findings-emnlp)

Challenge:	Multi-task learning (MTL) has been studied for sequence labeling tasks . auxiliary tasks are selected specifically to improve performance of a target task .
Approach:	They propose a shared-cell long-short-term memory cell which contains shared parameters that can learn from all tasks and task-specific parameters that could learn task-related information.
Outcome:	The proposed model can learn from all tasks and task-specific parameters.

Copied to clipboard

Junda Wu, Rui Wang, Tong Yu, Ruiyi Zhang, Handong Zhao, Shuai Li, Ricardo Henao, Ani Nenkova

Consistency Regularization for Cross-Lingual Fine-Tuning (2021.acl-long)

Challenge:	Existing deep learning models for sequence labeling are expensive and time-consuming.
Approach:	They propose an interactive sequence labeling that allows training directly with the user feedback . they identify context and feedback biases by formulating interactive sequence labels via a Structural Causal Model.
Outcome:	The proposed approach can effectively alleviate the biases and can be learnt with the user feedback.

Copied to clipboard

Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

Multilinguality

Challenge:	Experimental results show that consistency regularization improves cross-lingual fine-tuning . pre-trained cross-linguistic models can transfer task-specific supervision from one language to the other .
Approach:	They propose to improve cross-lingual fine-tuning with consistency regularization . they use example consistency regularized to penalize prediction sensitivity to four types of data augmentations .
Outcome:	The proposed method improves cross-lingual fine-tuning across tasks . it can be generalized to other target languages without additional training .

AiM: Taking Answers in Mind to Correct Chinese Cloze Tests in Educational Applications (2022.coling-1)

Copied to clipboard

Yusen Zhang, Zhongli Li, Qingyu Zhou, Ziyi Liu, Chao Li, Mina Ma, Yunbo Cao, Hongzhi Liu

Multimodality

Challenge:	Existing methods to correct handwritten assignments are to use OCR to recognize characters and compare them to answers.
Approach:	They propose a multimodal approach to correct handwritten Chinese characters by combining the visual information of students' handwriting with the encoded representations of answers.
Outcome:	The proposed model outperforms OCR-based methods by a large margin.

AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning (2024.naacl-long)

Copied to clipboard

Ruiyi Zhang, Rushi Qiang, Sai Ashish Somayajula, Pengtao Xie

VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation (2021.acl-long)

Challenge:	Large-scale pretraining followed by task-specific finetuning has achieved great success in various NLP tasks.
Approach:	They propose a meta learning based framework for automatically identifying the optimal rank of each LoRA layer.
Outcome:	The proposed framework is based on a meta learning based framework that can identify the optimal rank of each LoRA layer.

Copied to clipboard

Fuli Luo, Wei Wang, Jiahao Liu, Yijia Liu, Bin Bi, Songfang Huang, Fei Huang, Luo Si

Word Segmentation by Separation Inference for East Asian Languages (2022.findings-acl)

Challenge:	Existing work in multilingual pretraining relies on the shared vocabulary and bilingual contexts to encourage the correlation across languages.
Approach:	They propose to plug a cross-attention module into a Transformer encoder to explicitly build the interdependence between languages.
Outcome:	The proposed model outperforms existing models on XTREME and English-to-French translation datasets.

Copied to clipboard

Yu Tong, Jingzhi Guo, Jizhe Zhou, Ge Chen, Guokai Zheng

Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model (2021.emnlp-main)

Challenge:	Chinese Word Segmentation (CWS) is a sequence labeling task that divides sentences into words . despite diverse tagging schemas, they all carry implicit position information.
Approach:	They propose to model the separation state of every two consecutive characters by tagging them as two tags.
Outcome:	The proposed framework outperforms state-of-the-art on Japanese and Korean Word Segmentation datasets.

Copied to clipboard

Hongjiang Jing, Zuchao Li, Hai Zhao, Shu Jiang

Verb Metaphor Detection via Contextual Relation Learning (2021.acl-long)

Challenge:	Existing models focus on aspect term extraction, opinion term extraction and sentiment polarity classification but ignore the difference.
Approach:	They propose a joint aspect-based sentiment analysis task that focuses on the difference between the two tasks to improve the model's robustness.
Outcome:	Empirical results show that the proposed model outperforms the previous state-of-the-art on four benchmark datasets.

Copied to clipboard

Wei Song, Shuhui Zhou, Ruiji Fu, Ting Liu, Lizhen Liu

Towards Few-shot Entity Recognition in Document Images: A Label-aware Sequence-to-Sequence Framework (2022.findings-acl)

Challenge:	Recent work on verb metaphor detection focuses on analyzing restricted forms of linguistic context.
Approach:	They propose a model which explicitly models the relation between a verb and its various contexts.
Outcome:	The proposed model gets competitive results compared with state-of-the-art approaches on the VUA, MOH-X and TroFi datasets.

Copied to clipboard

Zilong Wang, Jingbo Shang

Benchmarking Approximate Inference Methods for Neural Structured Prediction (N19-1)

Challenge:	Entity recognition is a fundamental task in document image understandings.
Approach:	They propose to use label surface names to better inform a model of target entity type semantics and embed the labels into the spatial embedding space to capture spatial correspondence between regions and labels.
Outcome:	The proposed model can be built on a few shots of annotated document images . it can be used to better inform the model and capture spatial correspondence between regions .

Copied to clipboard

Lifu Tu, Kevin Gimpel

Sequence Repetition Enhances Token Embeddings and Improves Sequence Labeling with Decoder-only Language Models (2026.findings-eacl)

Challenge:	Structured prediction models often involve complex inference problems for which finding exact solutions is intractable.
Approach:	They propose to perform gradient descent with respect to the output structure directly and train a neural network to perform inference.
Outcome:	The proposed methods achieve better speed/accuracy/search error trade-off than gradient descent while being faster than exact inference at similar accuracy levels.

Copied to clipboard

Matija Luka Kukić, Marko Čuljak, David Dukić, Martin Tutek, Jan Šnajder

More Embeddings, Better Sequence Labelers? (2020.findings-emnlp)

Challenge:	Modern language models (LMs) are trained in autoregressive manner, conditioned on the prefix. sequence labeling (SL) tasks assign labels to each individual input token, naturally benefiting from bidirectional context.
Approach:	They explore sequence repetition (SR) as a less invasive alternative to decoder-only models . they show that increasing the number of repetitions does not degrade SL performance .
Outcome:	The proposed technique improves the quality of token-level embeddings and surpasses encoders and unmasked decoders.

Copied to clipboard

Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

Soft Language Clustering for Multilingual Model Pre-training (2023.acl-long)

Challenge:	Existing work suggests contextual embeddings improve sequence labeling accuracy . but, there is no definite conclusion on whether concatenating different kinds of embeddables is effective .
Approach:	They propose a family of contextual embeddings that improves sequence labeling accuracy . they conduct extensive experiments on 3 tasks over 18 datasets and 8 languages .
Outcome:	The proposed family of contextual embeddings improves the accuracy of sequence labelers over non-contextual embedders.

Copied to clipboard

Jiali Zeng, Yufan Jiang, Yongjing Yin, Yi Jing, Fandong Meng, Binghuai Lin, Yunbo Cao, Jie Zhou

Multilinguality

Challenge:	Multilingual pre-trained language models have demonstrated impressive (zero-shot) cross-lingual transfer abilities, however, their performance is hindered when the target language has distant typology from the source language or when pre-training data is limited in size.
Approach:	They propose a method that contextually retrieves prompts as flexible guidance for encoding instances conditionally.
Outcome:	The proposed method improves on the XTREME task and also for low-resource languages in unsupervised sentence retrieval.

4 and 7-bit Labeling for Projective and Non-Projective Dependency Trees (2023.emnlp-main)

Copied to clipboard

Carlos Gómez-Rodríguez, Diego Roca, David Vilares

Automatic Reconstruction of Missing Romanian Cognates and Unattested Latin Words (2020.lrec-1)

Challenge:	encodings that map trees into sequences of one discrete label per word have been proposed for constituency parsing and dependency parse.
Approach:	They propose an encoding that can represent any projective dependency tree as a sequence of 4-bit labels, one per word.
Outcome:	The proposed encoding achieves substantial accuracy gains over the previously best-performing sequence labeling encoders.

Copied to clipboard

Alina Maria Ciobanu, Liviu P. Dinu, Laurentiu Zoicas

Token-level Sequence Labeling for Spoken Language Understanding using Compositional End-to-End Models (2022.findings-emnlp)

Challenge:	Existing methods for producing related words are based on sequence labeling .
Approach:	They propose a method for producing related words based on sequence labeling . they aim to fill in gaps in incomplete cognate sets in Romance languages with Latin etymology and reconstruct uncertified Latin words.
Outcome:	The proposed method fills in gaps in incomplete cognate sets in Romance languages with Latin etymology and reconstructs uncertified Latin words.

Copied to clipboard

Siddhant Arora, Siddharth Dalmia, Brian Yan, Florian Metze, Alan W Black, Shinji Watanabe

Multimodality

Challenge:	End-to-end spoken language understanding systems model sequence labeling as a sequence prediction task causing a divergence from its well-established token-level tagging formulation.
Approach:	They propose to model sequence labeling as a sequence prediction task . their systems explicitly separate the added complexity of recognizing spoken mentions from the NLU task of sequence labelling .
Outcome:	The proposed systems outperform both cascaded and direct models on a labeling task of named entity recognition across SLU benchmarks.

Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models (2022.findings-emnlp)

Copied to clipboard

Lifu Tu, Caiming Xiong, Yingbo Zhou

Fine-grained Knowledge Fusion for Sequence Labeling Domain Adaptation (D19-1)

Challenge:	Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks.
Approach:	They do cross-lingual evaluation using prompt tuning and compare it with fine-tuning . prompt tuning achieves much better cross-linguistic transfer than fine- tuning .
Outcome:	The results show that prompt tuning achieves better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters.

Copied to clipboard

Huiyun Yang, Shujian Huang, Xin-Yu Dai, Jiajun Chen

Biomedical Event Extraction as Sequence Labeling (2020.emnlp-main)

Challenge:	Existing domain adaptation methods focus on the adaptation from the source domain to the entire target domain without considering the diversity of individual sample samples.
Approach:	They propose a fine-grained knowledge fusion model with the domain relevance modeling scheme to control the balance between learning from the target domain data and learning from a source domain model.
Outcome:	The proposed model outperforms baselines and state-of-the-art models on three sequence labeling tasks.

Copied to clipboard

Alan Ramponi, Rob van der Goot, Rosario Lombardo, Barbara Plank

Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning (2023.emnlp-main)

Challenge:	Empirical results show that BeeSL’s speed and accuracy makes it a viable approach for large-scale real-world scenarios.
Approach:	They propose a joint end-to-end neural information extraction model that recasts the task as sequence labeling and jointly models intermediate tasks via multi-task learning.
Outcome:	Empirical results show that BeeSL outperforms the current best system on the Genia 2011 benchmark by 1.57% absolute F1 score reaching 60.22% F1 .

Copied to clipboard

Sarkar Snigdha Sarathi Das, Haoran Zhang, Peng Shi, Wenpeng Yin, Rui Zhang

Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling (D19-1)

Challenge:	Named Entity Recognition, Relation Extraction, Semantic Role Labeling are examples of sequence labeling problems that require finetuning to the target format.
Approach:	They propose a dynamic sparse finetuning strategy that selectively focuses on a fraction of parameters, informed by feedback from highly regressing examples.
Outcome:	The proposed approach improves performance in low-resource settings and in extreme low-level settings.

Copied to clipboard

Xiaochuang Han, Jacob Eisenstein

An Exploration of Arbitrary-Order Sequence Labeling via Energy-Based Inference Networks (2020.emnlp-main)

Challenge:	Contextualized word embeddings are becoming a ubiquitous component of natural language processing.
Approach:	They propose a domain-adaptive fine-tuning approach to pretrain on unlabeled text . they test this approach on sequence labeling in two challenging domains .
Outcome:	The proposed approach improves on sequence labeling in two domains: Early Modern English and Twitter.

Copied to clipboard

Lifu Tu, Tianyu Liu, Kevin Gimpel

Challenge:	Recent work shows that conditional random fields (CRFs) perform well in sequence labeling tasks.
Approach:	They propose several high-order energy terms to capture dependencies among labels in sequence labeling . they use convolutional, recurrent, and self-attention networks to construct these energy terms .
Outcome:	The proposed approach improves on four sequence labeling tasks while having the same decoding speed as simple classifiers.

bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark (2023.acl-long)

Copied to clipboard

Momchil Hardalov, Pepa Atanasova, Todor Mihaylov, Galia Angelova, Kiril Simov, Petya Osenova, Veselin Stoyanov, Ivan Koychev, Preslav Nakov, Dragomir Radev

Sequence Labeling Parsing by Learning across Representations (P19-1)

Challenge:	bgGLUE is a benchmark for evaluating language models on natural language understanding (NLU) tasks in Bulgarian.
Approach:	They propose to use a benchmark to evaluate language models on NLU tasks in Bulgarian.
Outcome:	The proposed model performs well on sequence labeling tasks, but there is room for improvement for tasks that require more complex reasoning.

Copied to clipboard

Michalina Strzyz, David Vilares, Carlos Gómez-Rodríguez

Label-Agnostic Sequence Labeling by Copying Nearest Neighbors (P19-1)

Challenge:	Constituency and dependency parsing are the main abstractions for representing syntactic structure of sentences . constituency parsers are considered disjointed tasks, and their improvements have been obtained separately.
Approach:	They propose to add auxiliary loss to constituency parsing paradigms and explore a model that parses both paradigms at no cost.
Outcome:	The proposed model outperforms single-task models by 1.05 F1 points and 0.62 UAS points for constituency parsing and dependency parsers.

Copied to clipboard

Sam Wiseman, Karl Stratos

Information Retrieval

Challenge:	Retrieve-and-edit based structured prediction models condition on retrieved neighbors to generate new structures, but many models do not explicitly capture the discrete operations that allow for the neighbors to be edited into the target structure.
Approach:	They propose to explicitly condition on retrieved neighbors to create new structures . they propose to use a dynamic programming approach to sequence labeling .
Outcome:	The proposed model can perform accurate sequence labeling by explicitly copying labels from retrieved neighbors.

SmartSpanNER: Making SpanNER Robust in Low Resource Scenarios (2023.findings-emnlp)

Copied to clipboard

Min Zhang, Xiaosong Qiao, Yanqing Zhao, Shimin Tao, Hao Yang

Few-Shot Emotion Recognition in Conversation with Sequential Prototypical Networks (2021.emnlp-main)

Challenge:	Named Entity Recognition (NER) is one of the most fundamental tasks in natural language processing.
Approach:	They propose a method which introduces a Named Entity Head (NEH) prediction task to SpanNER and performs multi-task learning together with task of span classification.
Outcome:	The proposed method improves the robustness of SpanNER in low resource scenarios on the CoNLL03, Few-NERD, GENIA and ACE05 benchmark datasets.

Copied to clipboard

Gaël Guibon, Matthieu Labeau, Hélène Flamein, Luce Lefeuvre, Chloé Clavel

SpanNER: Named Entity Re-/Recognition as Span Prediction (2021.acl-long)

Challenge:	Existing studies on dyadic human-human interactions focus on conversations without specific business objectives.
Approach:	They propose a method to detect emotions in a live chat customer service . they propose 'ProtoSeq' for conversational emotion classification using different languages .
Outcome:	The proposed method is competitive even when applied to other ones.

Copied to clipboard

Jinlan Fu, Xuanjing Huang, Pengfei Liu

Unified Feature and Instance Based Domain Adaptation for Aspect-Based Sentiment Analysis (2020.emnlp-main)

Challenge:	Recent years have seen the paradigm shift of Named Entity Recognition (NER) systems from sequence labeling to span prediction.
Approach:	They experimentally implement 154 named entity recognition models on 11 datasets and show that span prediction can serve as a system combiner to re-recognize named entities from different systems’ outputs.
Outcome:	The proposed model can be used to re-recognize named entities from different systems’ outputs.

Copied to clipboard

Chenggong Gong, Jianfei Yu, Rui Xia

Sentiment Analysis

Challenge:	Existing approaches to aspect-based sentiment analysis rely on labeled data, but they lack the fine-grained labeles needed for the ABSA task.
Approach:	They propose a framework to perform feature adaptation and instance adaptation for the ABSA task . they learn domain-invariant feature representations by using part-of-speech features .
Outcome:	The proposed method improves on the state-of-the-art in two aspects of the ABSA task.

When Generative Adversarial Networks Meet Sequence Labeling Challenges (2024.emnlp-main)

Copied to clipboard

Yu Tong, Ge Chen, Guokai Zheng, Rui Li, Jiang Dazhi

Robustness in NLP

Challenge:	Existing approaches for sequence labeling use a feature extractor and sequence tagger . a recent study shows that SLGAN is versatile and highly effective .
Approach:	They propose a framework that harnesses the capabilities of Generative Adversarial Networks to address sequence labeling challenges.
Outcome:	The proposed framework exhibits strong adaptability to various sequence labeling tasks.

Chinese Metaphorical Relation Extraction: Dataset and Models (2023.findings-emnlp)

Copied to clipboard

Guihua Chen, Tiantian Wu, MiaoMiao Cheng, Xu Han, Jiefu Gong, Shijin Wang, Wei Song

Exploring and Predicting Transferability across NLP Tasks (2020.emnlp-main)

Challenge:	Metaphor identification is a core task in metaphor processing, which involves recognizing and analyzing metaphorical expressions in text.
Approach:	They propose a new formulation of metaphor identification as a relation extraction problem . they use a dataset to analyze metaphorical relations between two spans, a target and a source .
Outcome:	The proposed model can capture the properties of the target and source in Chinese sentences.

Copied to clipboard

Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer

Dependency Graph Parsing as Sequence Labeling (2024.emnlp-main)

Challenge:	Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks.
Approach:	They conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems.
Outcome:	The proposed model can improve performance even with low-data source tasks that differ substantially from the target task.

Copied to clipboard

Ana Ezquerro, David Vilares, Carlos Gómez-Rodríguez

FAME: Feature-Based Adversarial Meta-Embeddings for Robust Input Representations (2021.emnlp-main)

Challenge:	Various linearizations have been proposed to cast syntactic dependency parsing as sequence labeling, but they cannot handle reentrancy or cycles.
Approach:	They propose unbounded linearizations that can be used to cast dependency parsing as sequence labeling.
Outcome:	The proposed linearizations can cast syntactic dependency parsing as a sequence labeling task.

Copied to clipboard

Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow

To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning (2021.emnlp-main)

Challenge:	Recent work on word embeddings and pre-trained language models has shown the large impact of language representations on natural language processing (NLP) models across tasks and domains.
Approach:	They propose feature-based adversarial meta-embeddings with an attention function that is guided by word-specific properties, such as shape and frequency, to handle subword-based embeddings.
Outcome:	The proposed model improves performance in downstream tasks even with word embeddings from transformers.

Copied to clipboard

Lukas Lange, Jannik Strötgen, Heike Adel, Dietrich Klakow

Non-Autoregressive Machine Translation as Constrained HMM (2024.findings-acl)

Challenge:	Existing methods to select transfer sources are limited by text and task similarity, which limits their application in transfer settings where both the task and the text domain change.
Approach:	They propose a model similarity measure that represents text and task similarity jointly to automatically determine which and how many sources to exploit.
Outcome:	The proposed approach improves performance by 24 F1 points for predicting promising sources across domains and tasks with similar models.

Copied to clipboard

Haoran Li, Zhanming Jie, Wei Lu

Neuralizing Regular Expressions for Slot Filling (2021.emnlp-main)

Challenge:	Autoregressive (AR) models have some drawbacks due to slow inference speed and label bias due to local normalization.
Approach:	They propose to use a left-to-right Hidden Markov Model (HMM) to control label bias in non-autoregressive translation (NAT) They propose a bi-directional HMM, which can regularize each other's biases via shared parameters.
Outcome:	The proposed models can achieve comparable performance to autoregressive Transformers using various decoding methods.

Copied to clipboard

Chengyue Jiang, Zijian Jin, Kewei Tu

Semantic Parsing

Challenge:	Existing methods to integrate neural networks and symbolic rules have their merits and weaknesses.
Approach:	They propose to integrate regular expressions into neural networks for a slot filling task . they use finite-state transducers to convert regular expression into a neural network . their model has superior zero-shot and few-shot performance .
Outcome:	The proposed model outperforms rules in zero-shot and few-shot scenarios and is competitive when training data is available.

SeqVAT: Virtual Adversarial Training for Semi-Supervised Sequence Labeling (2020.acl-main)

Copied to clipboard

Luoxin Chen, Weitong Ruan, Xinyue Liu, Jianhua Lu

GNN-SL: Sequence Labeling Based on Nearest Examples via GNN (2023.findings-acl)

Challenge:	Empirical studies show that virtual adversarial training (VAT) significantly improves the sequence labeling performance over baselines under supervised and semi-supervised settings.
Approach:	They propose a method which naturally applies VAT to sequence labeling models with conditional random field (CRF) Empirical studies show that SeqVAT significantly improves the sequence labelling performance over baselines under supervised settings, and outperforms state-of-the-art approaches under semi-supervised settings.
Outcome:	Empirical results show that the proposed method outperforms state-of-the-art approaches under semi-supervised settings.

Copied to clipboard

Shuhe Wang, Yuxian Meng, Rongbin Ouyang, Jiwei Li, Tianwei Zhang, Lingjuan Lyu, Guoyin Wang

Multimodality

Challenge:	Existing sequence labeling algorithms can be decomposed into two parts .
Approach:	They propose a graph neural networks sequence labeling (GNN-SL) that augments the vanilla SL model output with similar tagging examples retrieved from the whole training set.
Outcome:	The proposed model performs well on three sequence labeling tasks.

Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling (2024.findings-acl)

Copied to clipboard

David Dukić, Jan Snajder